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PREFACE 


In  August  1983,  the  AFGWC  Real-Time  Nephanalysis  (RTNEPH)  Model  replaced  the  AFGWC  Automated  Cloud 
Analysis  Model  (3DNEPH).  This  study  evaluates  the  total  cloud  cover  (TCC)  variable  of  the  newer  RTNEPH. 

Many  AWS  customers  have  used  the  RTNEPH  TCC  without  knowing  its  true  accuracy.  One  such  customer  (a  HQ 
AWS  direct-reporting  unit)  tasked  the  USAFETAC  Special  Projects  Section  (ECS)  for  a  quantitative  evaluation  of 
the  TCC  variable. 

Before  developing  the  approach  used  in  the  study,  a  literature  search  was  conducted  with  the  help  of  the  Air  Weather  • 

Service  Technical  Library  (AWSTL)  to  examine  previous  work  on  the  subject.  When  little  work  of  any  scientific 
value  was  found,  USAFETAC/ECS  devised  the  approach  described  in  this  report  to  evaluate  the  accuracy  of  the 
RTNEPH  TCC  variable,  then  performed  and  documented  the  .study. 


Although  only  two  USAFETAC  sections  (Special  Projects  and  Environmental  Simulation)  were  directly  involved 
with  the  study,  special  recognition  is  given  to  the  AWSTL  and  the  Data  Automation  Branch  (AD)  for  their  extensive 
assistance. 
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1.  INTRODUCTION 

1.1  Purpose  of  the  Study.  The  purpose  of  this  study  is  to  evaluate  the  total  cloud  cover  (TCC)  variable  in  the 
Air  Force  Global  Weather  Center  (AFGWC)  automated  Real-Time  Nephanalysis  (RTNEPH)  cloud  model.  Because 
the  choice  of  global  databases  for  comparison  is  limited,  the  study  primarily  compares  RTNEPH-derived  TCCs  with 
those  of  nearby  independent  surface  observations.  A  point  study  that  uses  whole  sky  imager  (WSI)  data  from 
Malabar,  Florida,  is  included. 

1.2  Mods!  History.  In  1970,  the  United  States  Air  Force  implemented  a  program  to  map  and  store  (in  a  gridded 
database)  an  analysis  of  worldwide  cloud  cover.  This  analysis,  designated  "3-Dimensional  Nephanalysis" 
(3DNEPH)  (Fye,  1978),  contained  cloud-cover  amounts  for  IS  designated  height  intervals,  as  well  as  total  cloud 
cover.  The  3DNEPH  has  since  been  altered  to  improve  the  quality  of  the  cloud  database.  In  1983,  AFGWC 
replaced  the  3DNEPH  with  the  Real-Time  Nephanalysis  (RTNEPH)  model  in  an  effort  to  streamline  the  heavily 
modified  algorithm  and  more  accurately  specify  data  on  individual  cloud  layers  (Kiess  and  Cox,  1988).  As  part  of 
the  new  model,  diagnostic  flags  were  added  to  the  database;  to  specify  the  data  sources  used  in  the  processing  as 
well  as  the  age  of  the  latest  data.  The  grid  of  the  RTNEPH  ^tabase  is  polar  stereographic,  centered  on  the 
poles~see  Figures  1  and  2.  Spacing  between  grid  points  ranges  from  33  nm  to  13  nm  (pole  to  equator,  respectively) 
and  each  hemisphere  is  divided  into  64  boxes  containing  4,096  data  points  each.  Database  frequency  is  in  3  hourly 
increments  starting  with  OOZ.  In  reality,  RTNEPH  is  updated  on  receipt  of  new  data.  Snapshots  of  this  data  are 
taken  every  3  hours,  about  2  hours  after  data  valid  time. 

1.3  Previous  Studios.  Previous  research  efforts  on  RTNEPH  database  quality  are  limited  and  mostly 
unpublished.  A  few  studies  compared  3DNEPH/RTNEPH  and  surface  observations;  these  were  limited  in  scope, 
applied  only  to  a  few  selected  stations,  and  were  not  statistically  valid  because  the  RTNEPH  and  surface 
obsovadons  were  not  two  independent  datasets  (RTNEPH  incorporates  available  surface  observations  in  its 
analyses).  Other  studies  merely  focused  on  revealing  certain  strengths  and  weaknesses  of  RTNEPH.  Also,  most 
previous  articles  provide  only  qualitaUve  informaUon;  they  do  not  specify  the  level  of  accuracy  of  the  total 
cloud-cover  analysis.  For  these  reasons,  along  with  the  operaUonal  requirement  for  knowing  the  quantitative 
precision  of  the  RTNEPH  data,  USAFETAC  was  prompted  to  perform  this  large-scale  stadstical  study  using 
completely  independent  datasets. 
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NORtHCRN  HEMISPHERE 


Figure  1.  Northern  Hemisphere  RTNEPH  grid  over  a  polar-sterographic  projection. 

Comer  boxes  ate  not  used. 


2.  METHODS  OF  ANALYSIS 


To  evaluate  the  total  cloud-cover  dauiba.se,  we  compared  5  years  (1985-89)  of  RTNEPH  analy.ses  with  independent 
surface  observations.  Since  the  RTNEPH  algorithm  heavily  weights  available  surface  observations,  RTNEPH 
output  clo.scly  resembles  them.  To  conduct  a  suitistically  sound  study,  we  had  to  make  sure  that  the  surface 
observations  were  not  used  in  the  original  RTNEPH  model  prtKcssing  and  were  independent  of  the  RTNEPH 
analyses. 

We  did  this  by  choosing  isolated  synoptic  reporting  stations  that  did  not  have  any  other  active  reporting  suitions 
within  50  NM.  This  ensured  that  the  RTNEPH  model  was  not  using  data  from  other  stations  within  the  same  grid 
area  and  minimized  the  effect  of  RTNEPH  spreading  surface  data  into  the  grid  area  (nearest  the  surface  station)  from 
an  adjoining  grid  area. 

The  next  step  involved  extracting  the  RTNEPH  data  as  well  as  its  diagnostic  flags.  An  RTNEPH  observation  was 
chosen  if  the  diagnostic  flag  indicated  that  no  surface  .station  was  used  in  the  analysis,  or  even  if  surface  data  was 
used  but  was  over  3  hours  old.  Therefore,  the  corresponding  surface  observation  taken  at  c.,actly  the  same  time  as 
the  RTNEPH  analysis  was  expected  to  be  independent  of  the  RTNEPH  analysis  and  vice  versa.  This  allows 
verification  of  the  persistence  and  satellite  analysis  of  the  RTNEPH. 

Comparisons  of  the  RTNEPH  analyses  and  these  independent  surface  observations  were  then  sU'atified  and 
compared  by  latitude,  season,  time-of-day,  and  data  age  to  identify  the  strengths  and  weaknesses  of  the  RTNEPH 
analyses.  In  addition,  statistical  significance  tests  were  conducted  on  the  data  in  order  to  quantify  conclusions 
concerning  the  precision  of  the  RTNEPH  cloud-cover  analysis.  Since  RTNEPH-based  TCC  is  stored  as  whole 
percent  and  surface-based  TCC  (from  synoptic  reporting  stations)  is  reported  and  stored  in  octas  (eighths  of  sky 
coverage),  we  had  to  choose  a  standard.  We  converted  the  RTNEPH  data  into  octas  (using  WMO  standards),  since 
the  octa  is  the  least-precise  reportable  value;  conversion  of  octas  to  whole  percent  would  imply  more  precision  than 
exists.  More  than  420,000  such  observation  pairs  were  ased  in  the  analysis  from  168  surface  reporting  stations  (and 
their  corresponding  closest  RTNEPH  grid  points).  The  stations  were  selected  from  the  following  regions;  selection 
criteria  included  latitude  as  well  as  previous  USAFETAC  data  requests  for  TCC  data;  see  Figures  1  and  2  for 
RTNEPH  box  locations.  An  added  value  of  100  indicates  boxes  in  the  southern  hemisphere. 


Latitude 

Geographic  Region 

RTNEPH  Box  # 

Polar 

Northern  USSR/Finland 

29 

Mid-Latitude 

Korea/Eastem  China 

20 

Mid-Latitude 

Iraq/Turkey/Syria 

22 

Mid-Latitude 

Chile/ Argentina 

121 

Mid-Latitude 

New  Zealand 

135 

Tropical 

Western  Africa 

40 

Tropical 

Eastern  Pacific  Islands 

60 

Tropical 

Colombia/Venezuela 

61 
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3.  ASSUMPTIONS  AND  LIMITATIONS  OF  THE  DATA 


3.1  Ground  Truth.  The  assumption  was  made  that  (he  surface  observations  represent  "ground  truth."  Since 
clouds  are  between  1  and  2  orders  of  magnitude  closer  to  ground  observers  than  they  are  to  the  polar-orbiting 
meteorological  satellites  that  feed  the  RTNEPH  model,  the  resulting  determination  of  cloud  cover  doesn’t  always 
agree.  Figure  3  shows  the  inherent  differences  in  viewing  perspectives  between  earth-  and  space-based 
observations.  For  the  surface  observer,  clouds  closer  to  the  horizon  increase  the  apparent  sky  cover  because  the 
sides  of  the  clouds  (not  just  their  bases)  obscure  the  celestial  dome.  For  space-based  viewing,  the  cloud  scenes  are 
projected  onto  a  flat  surface;  at  ground  level,  they  are  projected  unto  a  curved  surface.  In  reality,  the  satellite  and 
surface  sky  cover  amounts  each  represent  a  different  physical  phenomenon.  A  rough  correction  can  be  made  to  the 
surface  observation  if  one  knows  the  heights  and  angular  location  of  the  clouds  (Snow,  1990).  Since  this 
information  is  not  recorded  by  observers,  we  are  unable  to  correct  for  it.  The  net  effect  is  that  surface-based 
observations  should,  on  average,  be  cloudier  than  the  corresponding  space-based  observations.  However,  since 
surface  data  is  reported  to  the  nearest  octa,  this  effect  should  be  .small.  It  is  reasonable  to  assume  that  the  surface 
observing  bias  will  rarely  be  greater  than  one  octa,  and  in  most  cases,  less. 


Figure  3.  Cloudiness  quantifications.  S,  apparent  cloud  cover;  C,  cloud  cover;  C,  ground  cover;  N,  sky 
cover.  ^  is  view  angle.  Heavy  line  segments  depict  areas  masked  from  view  by  cloud  (Snow,  1990). 

3.2  Observer  Subjectivity.  The  subjectivity  of  the  observer  also  contributes  to  error.  Differing  skill  and 
experience  levels  of  human  observers,  as  well  as  differences  in  repotting  practices  around  the  world,  result  in 
subjectivity  and  inconsistency.  In  this  study,  we  are  combining  large  numbers  of  observations  from  various 
locations.  This,  along  with  random  sampling,  should  cancel  out  observer-induced  biases.  An  exception  is  the 
nighttime  observational  bias,  which  one  would  expect  to  be  common  to  all  locations.  Various  studies  have  reported 
that  observers  tend  to  under-report  cloud  cover  at  night;  that  is,  mean  cloud-cover  on  nights  with  a  full  moon  can 
vary  up  to  20%  from  nights  with  a  new  moon.  By  stratifying  the  data  day/night,  we  should  be  able  to  determine  the 
significance  of  this  bias. 

3.3  Diagnostic  Flags.  The  assumption  is  also  made  that  (he  RTNEPH  diagnostic  flags  accurately  show  whether 
or  not  surface  reports  were  processed  for  a  particular  analysis.  While  this  is  a  significant  assumption,  examination  of 
TCC  frequency  distributions  of  both  the  RTNEPH  and  surface  reports  from  various  selected  stations  lead  us  to 
believe  the  flags  are  operating  properly. 

3.4  Bogusing  Effects.  Bogusing  refers  to  the  manual  subjective  adju.stment  of  the  TCC  amount  by  AFGWC 
analysts  if  a  visual  inspection  of  the  satellite  Imagery  appears  to  conflict  with  the  computer  analysis  (Kiess  and  Cox, 
1988).  Bogusing  tends  to  smooth  an  area  of  clouds  that  may  or  may  nor  be  "good."  It  is  unknown  where  or  when 
ix^using  occurred,  but  AFGWC  suggests  most  is  done  in  the  mid-latitudes.  We  should  therefore  expect  the  best 
agreement  between  RTNEPH  and  the  surface  observer  in  this  latitude  band.  In  any  case,  bogusing  is  an  integral  part 
of  the  RTNEPH  analysis  and  resulting  database;  it  should  only  improve  an  analysis. 
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4.  STRATIFICATIONS 


4.1  Stratification  by  Latitude  Band.  In  order  lo  provide  a  thorough  study  of  the  database,  we  stratified  the 
data  into  several  categories.  The  initial  stratification  divided  the  global  station  coverage  into  different  latitudinal 
bands.  The  breakdown  of  the  bands  is  as  follows:  Polar  Latitudes  (poleward  of  70"),  Mid-Latitudes  (30"  -  60°),  and 
Tropical  Latitudes  (15°  S  -  15°  N).  The  regions  were  broken  down  this  way  because  of  known  RTNEPH 
deHciencies,  such  as  ice/cloud  distinctions  in  arctic  areas  and  .satellite  data  underlap  in  tropical  areas. 

4.2  Season  Stratification.  The  second  stratification  was  by  season  (winter,  spring,  summer,  and  autumn).  This 
breakdown  was  accomplished  for  each  region  in  addition  to  the  combined  or  global  data  set. 

4.3  Day/NIght  Stratification.  The  third  stratification  involved  grouping  the  observations  by  day  and  night.  For 
this  study  we  considered  a  mid-latitude  location,  arbitrarily  selecting  RTNEPH  Box  22  (Middle  East).  We  examined 
only  stations  in  adjoining  time  zones  to  ensure  they  all  were  experiencing  daylight  or  nighttime  conditions  for  the 
pre-set  hours.  Six-hour  intervals  were  studied  in  each  case.  The  hours  of  ()900-15(X)L  (()600-12(X)Z)  depicted 
daylight,  while  21()0-03()0L  (ISOO-OBOOZ)  depicted  nighttime.  These  time  intervals  were  chosen  to  avoid  the  optical 
effects  of  dawn  and  dusk.  Although  we  limited  this  stratification  to  one  region,  we  anticipate  a  high  global 
consistency  of  day/hight  bias  applies  to  non-arctic  regions.  Day  and  night  stratification  in  the  arctic  regions  was  not 
done  because  no  practical  meteorological  benefit  was  expected  due  to  the  summer  day  and  winter  night  tendencies. 

4.4  Data  Age  Stratification.  The  final  stratification  was  for  "data  age,"  which  refers  to  the  difference  between 
the  latest  data  observation  time  (from  whatever  source-satellite,  surface,  aircraft,  etc.)  and  the  RTNEPH  reference 
time  (the  official  time  of  the  analysis).  Because  of  the  non-symmetrical  meteorological  satellite  constellation,  and 
because  of  the  highly  latitude-dependent  characteristics  of  polar  orbits,  data  age  varies  greatly.  We  limited  this 
study  to  data  up  to  24  hours  old,  since  most  falls  within  this  category.  In  addition,  this  test  was  only  accomplished 
for  the  global  stratification.  We  stratified  data  age  with  the  following  lime  categories:  0-3  hours,  4-6  hours,  7-12 
hours,  13-18  hours,  and  19-24  hours. 
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5.  ANALYSIS 


5.1  Regional  and  Seasonal  Stratifications.  The  first  two  slraiincalions  (region  and  season)  were  combined 
for  the  analysis.  The  global  data  set,  and  those  of  the  individual  regions,  were  broken  down  by  season.  For  this 
study,  we  examined  both  the  whole  population  and  random  samples  of  5,(X)0.  U.sc  of  multiple  random  samples 
provides  better  confidence  in  the  rcsulLs  of  statistical  tests  and  ensures  our  conclusions  arc  robust.  (A  robust 
estimate  is  one  that  is  not  restricted  to  a  particular  sample  or  dependent  upon  a  key  assumption.)  Figures  4-7  give 
cloud-cover  frequency  distributions  for  the  population  as  well  as  two  random  samples  of  5,000  for  RTNEPH  reports 
in  the  polar  latitudes.  These  figures  and  statistical  tests  show  that  this  sample  size  is  sufficient  to  represent  the 
overall  population. 
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RTNEPH  SKY  COVER  DISTRIBUTION  IN  OCTAS 


Rgura  4.  Difference  Between  RTNEPH  Reports  for  Different  Sample  Sizes,  Polar 
LatKudea,  Winter. 


7 


O  U. 


60 


0  1  2  3  4  5  6  7  8 
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Figure  5.  Difference  Between  RTNEPH  Reports  for  Different  Sample  Sizes,  Polar 
Latitudes,  Spring. 
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Figure  6.  Difference  Between  RTNEPH  R^rts  for  Different  Sample  Sizes,  Polar 
Latitudes,  Summer. 
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Figure  7.  Difference  Between  RTNEPH  Reports  for  Different  Semple  Sizes,  Polar  Latnudes, 
Autumn. 


Frequency  distributions  of  die  differences  between  the  paired  values  (RTNEPH  value  minus  the  surface  observation 
value)  were  generated  fw  each  stratification.  As  mentioned  in  Section  2,  the  surface  observations  were  recorded  in 
synoptic  code,  offering  a  resolution  in  octas.  The  RTNEPH  percentage  values  were  therefore  converted  to  octas 
usii%  WMO  conversion  standards. 


After  the  distributions  were  calculated  for  the  global  and  regional  databases,  the  results  were  combined  and  plotted 
on  a  single  gnqih  for  each  season  (Figures  8-11).  As  expected,  each  distribution  has  a  well-pronounced  peak  at  the 
zero  interval.  The  percentage  of  observations  within  1  and  2  octas  of  each  other  are  summarized  in  Tables  I  and  2, 
respectively.  These  percentages  quantify  the  precision  of  die  RTNEPH  analysis. 
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TABLE  1.  Percent  of  RTNEPH  Total  Cloud  Cover  Observation  within  1  Octa  (by  Region). 


SERSON 

Global 

Polar 

Mid-Lat 

Tropical 

WINTER 

59 

47 

67 

57 

SPRING 

58 

49 

65 

54 

SUMMER 

64 

58 

69 

58 

AUTUMN 

64 

58 

70 

60 

TABLE  2.  Percent  of  RTNEPH  Total  Clouds  Cover  Observation  within  2  Octas  (by  Region). 


SEASON 

Global 

Polar 

Mid-Lat 

Tropical 

WINTER 

69 

59 

76 

71 

SPRING 

69 

60 

75 

68 

SUMMER 

74 

68 

79 

69 

AUTUMN 

74 

68 

79 

71 
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RTNEPH  -  SURFACE  SKY  COVER  IN  OCTAS 
FIgura  8.  Latitude  Comparleon  of  RTNEPH  minus  Surface  Observations,  Winter. 
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RTNEPH  -  SURFACE  SKY  COVER  IN  OCTAS 
Figure  9.  Latltucte  Comparison  of  RTNEPH  minus  Surface  Observations,  Spring. 
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Figure  10.  Latitude  Comparison  of  RTNEPH  minus  Surface  Observations,  Summer. 
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As  shown  in  Tables  1  and  2  and  Figures  8-1 1,  the  surface  observations  and  the  RTNEPH  agree  most  of  the  time. 
The  largest  discrepancy  occurs  in  the  polar  regions,  especially  during  winter  and  spring.  Note  the  small,  persistent 
peak  over  the  -7  interval.  (A  negative  difference  is  the  result  of  the  observer  recording  more  cloud  cover  than  the 
corresponding  RTNEPH  analysis).  Persistent  ice  or  snow  cover  and  low  suatus  are  often  indistinguishable  in 
infirar^  satellite  imagery  (visual  data  is  not  used  over  ice-  or  snow-covered  areas)  due  to  lack  of  temperature  and 
background  color  contrast;  this  may  well  be  the  source  of  these  errors.  As  a  result,  the  polar  discrepancy  affects  the 
global  distribution. 
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RTNEPH  -  SURFACE  SKY  COVER  IN  OCTAS 
Figure  11.  Latitude  Comparison  of  RTNEPH  minus  Surface  Observations,  Autumn. 
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5.2  Day/NIght  Stratification.  Nighllimc  comlilions  alTcct  bolh  ihc  salcllilc’s  and  ground-based  observer’s 
abilities  to  determine  cloud  cover  accurately.  To  evaluate  this  potential  bias,  an  area  in  the  mid-latitudes  (RTNEPH 
Box  22)  was  selected;  we  applied  the  same  dirfcrcncing  technique  lietwccn  the  RTNEPH  and  surlacc  observations  as 
discussed  earlier.  It  should  be  noted  that  RTNEPH  was  updated  early  in  the  day /night  pcrioil  (by  1(XX)/22(X)L)  with 
DMSP  satellite  passes;  the  rest  of  the  period  represents  persistence.  No  matter  what  region  of  the  world  is  chosen, 
each  day/night  period  would  only  be  updated  once  (except  for  polar  regions),  and  the  rest  of  the  day/night  period 
would  represent  persistence. 
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SKY  COVER  (RTNEPH  -  SURFACE)  IN  OCTAS 

Figure  12.  Comparison  of  Day/NIght  RTNEPH  minus  Surface  Observations,  mId-Latitudes,  Winter. 
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Separate  plots  were  prepared  for  daytime  and  nighttime  observations.  Figures  12-15  depict  the  seasonal 
distributions  for  this  stratiHcation.  They  indicate  more  agreement  at  night  than  during  the  day.  We  su.spect  that  this 
is  just  as  likely  to  indicate  that  RTNEPH  and  surface  reports  are  similarly  biased  by  the  disabling  effects  of  darkness 
(the  satellite  has  no  visible  imagery  at  night)  as  it  is  to  imply  that  both  are  more  accurate  at  night.  Although  both 
show  more  clear  observations  at  night,  this  does  have  physical  support;  convective  clouds  often  dissipate  soon  after 
dark.  Frequency  distributions  were  plotted  for  both  the  RTNEPH  and  surface  observations  comparing  the  day 
versus  night  hours  (Figures  16-19  and  20-23). 
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FIgura  13.  Comparison  of  Day/NIght  RTNEPH  minus  Surfacs  Observations,  mId-LatItudes,  Spring. 
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SKY  COVER  (RTNEPH  -  SURFACE)  IN  OCT  AS 


Figure  14.  Comparison  of  Day/NIght  RTNEPH  minus  Surface  Observations, 
mkl-Latitudes,  Summer. 
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SKY  COVER  (RTNEPH  -  SURFACE)  IN  OCT  AS 


Figure  15.  Comparison  of  Day/NIght  RTNEPH  minus  Surface  Observations,  mld-Latitudes, 
Autumn. 
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Figure  16.  Comparison  of  Day/Night  Stratification  for  RTNEPH,  MkJ-Latitudas,  Winter. 
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Figure  17.  Comparleon  of  Day/Nlgfrt  Stratification  for  RTNEPH,  Mid-Latitudes,  Spring. 
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SKY  COVER  IN  OCTAS 


Figure  18.  Comparison  of  Day/NIght  Stratification  for  RTNEPH,  Mid-Latitudes,  Summer. 
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SKY  COVER  IN  OCTAS 


Figure  21 .  Comparlton  of  Oay/Nigin  Stratification  tor  Surface  Databases,  Mltl-Latltutfes,  Spring. 
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Figure  22.  Comparlaon  of  Day/NIgfit  Stratification  for  Surface  Databases,  Mid-Latitudes,  Simimer. 
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5.3  Timo-Lag  Comparison.  The  global  RTNEPH  minus  surface  TCC  differences  were  grouped  by  data  age  as 
descnbed  in  Section  4.4  and  slratified  by  season.  The  resultant  frequency  distributions  for  the  time-lag  hours  are 
plotted  in  Figures  24-27.  The  RTNEPH  TCC  amounts  did  not  show  significant  differences,  even  in  the  19-24  hour 
group.  The  percentage  of  observations  with  zero  error  is  smallest  in  the  4-6  hour  range;  it  actually  increases  for  the 
19-24  hour  range  to  nearly  the  value  for  the  0-3  hour  range.  This  is  conu^ry  to  expectation,  but  it  may  be  the  result 
of  diurnal  effects  such  as  low  clouds,  which  dissipate  during  the  day  but  often  return  again  the  same  time  the 
following  day.  Also,  a  significant  fraction  of  the  ±  7  and  ±  8  errors  are  in  the  19-24  hour  interval.  This  is  more  in 
line  with  expectation.  All  this  data,  however,  may  have  been  affected  by  the  bogusing  discussed  in  Section  3.4.  We 
don’t  know  to  what  extent  bogusing  changed  the  older  data. 
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SKY  COVER  (RTNEPH  •  SURFACE)  IN  OCTAS 

FIgurt  24.  Comparison  of  RTNEPH  and  Surfaca  Raport  DHtorancaa  for  omarant 
TMwLag  Hour  Qroupa,  AM  Lamudao,  Wimar. 
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Rgure  25.  Comparison  of  RTNEPH  and  Surface  Report  Differences  for  DKferem 
Thne-Lag  Hour  Groups,  All  Latitudes,  Spring. 


ooo  "no  Oman 


SKY  COVER  (RTNEPH  -  SURFACE)  IN  OCTAS 

Figure  26.  Comparison  of  RTNEPH  and  Surface  Report  Differences  for  Different 
Time-Lag  Hour  Groups,  All  Latitudes,  Summer. 
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Figure  27.  Comparison  of  RTNEPH  and  Surface  Report  Differences  for  Different 
Timo-Lag  Hour  Groups,  Ail  Latitudes,  Autumn. 
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6.  STATISTICAL  TESTS 

Three  statistical  tests  were  used  to  evaluate  the  differences  between  the  RTNEPH  and  surface  cloud-cover 
observations.  The  test  results  can  be  used  to  provide  an  estimate  of  reliability  for  the  TCC  amount  in  the  RTNEPH 
analysis.  We  used  the  t-test,  the  chi-square  test,  and  the  Spearman  correlation  coefncienl. 

6.1  T'Test.  Most  statistical  tests  are  based  on  the  assumption  that  data  are  normally  distributed;  cloud  cover 
distributions,  however,  are  not.  According  to  Box  et  al.  (1978),  the  t-test  is  insensitive  to  the  normality  assumption. 
When  the  distribution  is  non-normal,  the  distribution  of  the  sample  means  will  not  be  exactly  normal.  However,  due 
to  the  central  limit  theorem,  the  distribution  of  the  sample  means  will  tend  toward  normality  as  the  sample  size 
increases.  Box  et  al.  demonstrated  that  the  significance  level  of  the  t-test  is  barely  affected  by  dramatic  changes  in 
the  parent  distribution.  Hays  and  Winkler  (1971)  further  pointed  out  that  while  the  derivation  of  the  t-distribution 
requites  the  assumption  of  normality,  in  practice,  the  t-distribution  may  be  applied  when  the  parent  distribution  is 
non-normal,  provided  that  the  random  sample  size  is  moderately  large.  Since  this  study  uses  random  samples  of 
5,(X)0,  we  believe  the  results  of  the  t-test  are  valid  and  require  no  caveat. 

Tables  3-5  summarize  the  results  of  each  test  The  t-test  evaluates  the  null  hypothesis  that  the  mean  difference 
between  the  RTNEPH  and  surface  TCC  amounts  is  zero.  This  test  does  not  evaluate  the  quality  of  any  randomly 
selected  observation;  instead,  it  assesses  the  agreement  of  the  data  as  a  whole.  If  the  mean  difference  is  zero,  then 
the  sign  of  the  difference  between  any  given  observation  pair  should  be  randomly  plus  or  minus;  i.e.,  the  difference 
is  unbiased.  If  the  t-test  rejects  the  null  hypothesis,  it  implies  a  bias  in  the  differences.  The  accepi/reject  threshold 
was  based  on  a  5%  level  of  signiHcance.  For  the  mid-latitudes  and  tropics,  in  each  season  except  winter  in  the 
tropics,  the  null  hypothesis  was  accepted.  During  winter  in  the  tropics  and  for  all  four  seasons  at  the  polar  latitudes, 
the  null  hypothesis  was  rejected,  implying  a  bias  in  the  data. 


TABLE  3.  Polar  Latitude  Statistical  Test  Results  (by  Season). 


SERSON 

T-TEST 

CHI-SQURRED 

PHI  COEF 

CORRELATION 

COEFFICIENT 

UINTER 

Reject 

Different 

0.32 

0.30 

SPRING 

Reject 

Different 

0.35 

0.27 

SUMMER 

Reject 

Different 

0.43 

0.34 

AUTUMN 

Reject 

Different 

0.35 

0.35 
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TABLE  4.  Mid-Latitude  Statistical  Test  Results  (by  Season). 


SERSON 

T-TEST 

CHI-SQUARED 

PHI  COEF 

CORRELATION 

COEFFICIENT 

WINTER 

Accept 

Different 

0.17 

0.60 

SPRING 

Accept 

Different 

0.19 

0.63 

SUMMER 

Accept 

Different 

0.17 

0.71 

AUTUMN 

Accept 

Different 

0.14 

0.67 

TABLE  5.  Tropical  Latitude  Statistical  Test  Results  (by  Season). 


SEASON 

T-TEST 

CHI-SQUARED 

PHI  COEF 

CORRELATION 

COEFFICIENT 

WINTER 

Reject 

Different 

0.20 

0.52 

SPRING 

Accept 

Different 

0.22 

0.50 

SUMMER 

Accept 

Different 

0.10 

0.52 

AUTUMN 

Accept 

Different 

0.12 

0.51 

6.2  Chi-Square  Test.  The  chi-square  lest  is  used  to  evaluate  the  null  hypothesis  that  two  frequency  distributions 
are  similar.  Brooks  and  Carruthers  (1953)  state  that  any  two  observed  disuibulions  may  be  compared  with  the 
chi-square  test  without  reference  to  the  theoretical  forms  of  the  disuibutions.  They  further  slate  that  a  useful 
application  of  the  test  in  meteorology  is  testing  the  similarity  of  frequency  distributions  of  two  or  more  sets  of 
observations.  In  this  case,  we  compared  the  frequency  distributions  of  cloud  cover,  derived  from  the  RTI^EPH 
analysis,  with  surface  observations.  In  all  cases,  at  the  5%  level  of  significance,  the  null  hypothesis  was  rejected. 
Thus  the  distributions  are  statistically  different 

Because  the  chi-square  test  is  very  sensitive  to  sample  size,  and  to  evaluate  the  degree  to  which  the  distributions 
differ,  we  use  the  phi  coefficient  (the  square  root  of  the  chi-square  statistic  divided  by  the  sample  .size).  This 
coefficient  ranges  from  -1  to  +1.  Values  close  to  zero  indicate  the  distributions  are  nearly  identical,  while  those 
approaching  either  1  or  -1  are  significantly  distinct,  in  most  of  the  cases,  the  phi  coefficient  was  less  than  or  equal  to 
0.20,  implying  a  large  degree  of  similarity  in  the  two  distributions.  However,  at  the  polar  latitudes,  values  ranged 
from  0.32  to  0.43.  This  suggests  there  are  some  noticeable  differences  between  the  RTNEPH  and  surface  data  in  the 
polar  latitudes. 
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6.3  Spearman  Correlation  Coefficient  Test.  The  final  test  was  ihe  Spearman  correlation  coefficient,  a 
non-paramctric  test  (Conover,  1990)  that  determines  how  well  the  individual  observations  are  correlated.  The  more 
familiar  form  of  the  correlation  coefficient  is  the  Pearson  product  moment,  which  was  not  used  because  it  is  based 
on  an  assumption  of  normality  and  is  too  sensitive  to  non-normality. 

The  values  for  the  Spearman  coefficient  also  range  from  -1.0  to  1.0.  In  all  cases,  the  samples  are  positively 
correlated  with  coefficient  values  hovering  around  0.5.  The  mid-latitudes  performed  best,  with  correlation  values 
ranging  from  .60  to  .71  depending  on  the  season.  Once  again,  the  polar  region  did  not  have  as  favorable  results  as 
the  other  latitudinal  regions. 

6.4  Test  Conclusions.  Tables  3-3  indicate  that  seasons  do  play  a  significant  role  in  the  accuracy  of  the 
RTNEPH  total  cloud-cover  analysis.  The  two  sources  are  in  closest  agreement  during  the  summer  months.  The  fact 
that  the  greatest  problems  in  the  data  occur  at  the  polar  latitudes  suggests  that  the  RTNEPH  analysis  does  in  fact 
have  difficulty  separating  ice/snow  cover  from  cloud-cover.  In  the  tropics,  winter  differences  reveal  a  positive  bias 
(RTNEPH  TCC  greater  than  surface  TCC).  This  may  be  a  result  of  small-scale  cloud.^.  If  small-scale,  evenly 
distributed  clouds  are  present,  they  can  inundate  the  coarser  pixels  comprising  the  RTNEPH  grid  area,  resulting  in 
an  overestimate  of  TCC.  In  any  event,  the  observed  bias  is  small,  averaging  only  about  1  octa. 

Overall,  the  test  results  are  consistent.  Three  fundamental  tests  were  conducted  to  evaluate  different  aspects  of  the 
comparison  between  the  RTNEPH  and  surface  total  cloud-cover  reports.  They  point  to  good  meteorological 
agreement  for  TCC  between  the  two  databases.  We  must  u<ie  caution,  however,  because  statistics  can  often  be 
misleading  even  when  using  large  sampling  sizes.  For  example,  if  we  collected  cloud  data  at  a  location  for  one 
complete  year  and  examined  it  against  cloud  data  for  the  same  location  the  following  year,  we  would  probably 
conclude  that  the  data  was  from  different  locations.  The  reason  for  this  is  the  large  year-to-year  variability  in 
cloud-cover  that  occurs  at  any  one  location. 
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7.  WHOLE  SKY  tMAGERY  COMPARISON 


To  conclude  our  study  of  the  TCC  analysis,  we  compared  it  with  whole  sky  imagery  (WSl)  data.  The  Phillips 
Laboratory  Geophysics  Directorate  provided  WSI  data  for  Malabar,  Florida,  5  miles  NE  of  Melbourne  Regional 
Airport.  The  data  was  only  available  from  6  October  1989  to  3  April  1990  for  6  hours  either  side  of  local  noon. 
Surface  observations  were  obtained  from  Melbourne  Regional  Airport.  Because  of  the  limited  amount  of  WSI  data, 
this  point  study  is  only  included  to  point  out  a  weakness  in  the  RTNEPH  model  analysis  over  countries  such  as  the 
United  States. 

For  this  part  of  the  study,  RTNEPH  data  was  not  independent  from  surface  observations,  and  we  expected  very  high 
agreement  between  them.  The  RTNEPH  and  WSI  data  were  independent.  Figures  28-30  show  the  results  for  three 
seasons-'Summer  is  not  included.  Analysis  of  the  surface  observations  and  RTNEPH  data  (from  the  grid  area  closest 
to  the  WSI  sensor)  clearly  showed  the  model  directly  used  surface  reports  from  Melbourne  Regional  Airport  for  this 
grid  area;  therefore,  it  is  not  surprising  that  the  RTNEPH  and  surface  observations  compared  favorably.  This  was 
not  the  case  for  RTNEPH  and  WSI  data  for  the  same  time  period.  Because  the  airways  reporting  code  (not  synoptic) 
is  used  at  Melbourne,  TCC  resolution  was  degraded  from  octas  to  one  of  four  repotting  categories:  clear,  scattered, 
broken,  or  overcast. 
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SKY  COVER  (RTNEPH  -  SURFACE  OBS  AND  RTNEPH-WSI)  IN  OCTAS 
Figure  28.  ComfMirleon  of  RTNEPH  WSI  and  Surface  Reports  Over  Malabar,  FL,  Winter. 
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Figure  29.  Comparison  of  RTNEPH  WSI  and  Surface  Reports  Over  Malalrer,  FL,  Spring. 
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Figure  30.  Comparteon  of  RTNEPH  WSi  and  Surface  Reports  Over  Malabar,  FL,  Autumn. 


WSl  imagery  is  recorded  as  a  percentage  of  sky  cover.  Since  the  RTNEPH  relies  heavily  on  surface  observations, 
nearly  all  cloud-cover  amounts  are  either  0,  25,  75,  or  100%.  As  a  result,  the  RTNEPH- WSl  comparison  was  poor, 
implying  that  RTNEPH  total  cloud-cover  amounts  over  the  United  Slates  (and  other  areas  with  widespread  airways 
code  reporting)  have  poor  resolution.  Since  the  other  stralifications  showed  the  satellite  to  be  a  reliable  source, 
it  might  be  advantageous  for  RTNEPH  to  record  cloud-cover  over  the  U.S.  using  only  satellite  data  (especially  if 
surface-based  observed  data  is  other  ihiui  clear  or  overcast).  However,  one  must  first  consider  the  persistence  of  the 
saiellile-derived  observations  against  the  coarser  resolution  of  airways  observations. 


I  I  SFCOBS 

WSl 

m 

1 

_ L 

u 

III 

iiniifH 

.8  -7  -6  -5  -4  -3  -2  -1  0  -H  +2  +3  +4  -t5  +6  +7  +8 

SKY  COVER  (RTNEPH  -  SURFACE  OBS  AND  RTNEPH-WSI)  IN  OCT  AS 


38 


8.  CONCLUSIONS 


Independent  observations  of  total  cloud-cover  made  from  surface-based  observers  show  that  satellite-derived 
cloud-cover  amounts  in  the  RTNEPH  analysis  have  a  large  degree  of  precision.  The  largest  discrepancies  appear  in 
polar  regions,  where  the  RTNEPH  analysis  tends  to  underestimate  observed  cloud-cover,  probably  due  to  difriculties 
in  discriminating  between  low  cloud  and  snow-  or  ice-covered  ground.  A  major  revision  in  the  RTNEPH  algorithm 
is  scheduled  for  implementation  in  1991;  the  changes  include  the  use  of  microwave  channels  for  estimating  surface 
temperatures.  We  believe  that  this  and  other  changes  may  significantly  improve  detection  of  clouds  over 
snow-covered  surfaces.  Actually,  microwave  capability  for  estimating  surface  temperauires  has  been  available  since 
1989,  but  it  was  turned  off  due  to  problems  with  SSM/I  data.  Future  DMSP  satellites  should  have  those  problems 
corrected,  and  with  RTNEPH  stratifying  the  data  by  day^ight/snow/land/water/ice/satellite,  they  should  provide 
better  information  for  delecting  clouds.  A  small  bias  of  over-reporting  of  cloud-cover  was  also  noted  for  the  tropical 
latitudes  during  the  winter  months. 

The  effect  of  the  age  of  the  RTNEPH  data  produced  unexpected  results.  The  poorest  comparisons,  on  average,  were 
associated  with  data  4-6  hours  old.  Older  data  improved  slightly  with  age,  with  the  19-24  hour  group  having  a 
similar  frequency  of  a  zero  difference  (RTNEPH  minus  surface)  as  for  the  0-3  hour  group.  We  attribute  this  to 
diurnal  effects.  The  19-24  hour  range  did  show  the  greatest  frequency  for  the  absolute  value  of  this  difference  being 
greater  than  or  equal  to  7  octas.  Overall,  however,  data  age  appears  to  be  a  minor  problem. 

Next  we  examined  day  and  night  observations  for  evidence  of  bias.  It  appears  that  an  observer  is  more  likely  to 
rqxMt  clear  skies  at  night  when,  in  fact,  scattered  clouds  are  detected  by  the  RTNEPH  analysis.  However,  the 
overall  difference  in  the  two  cloud-cover  amounts  is  generally  2  octas  or  less.  The  results  show  that  RTNEPH  and 
the  surface  observer  both  suffer  disabling  effects  at  night  Both  report  more  clear  conditions  and  they  agree  with 
each  other  very  well  at  night 

Research  data  sets  such  as  the  whole  sky  imagery  data  are  good  sources  of  objective  cloud-cover  data,  and  would  be 
ideal  for  evaluating  RTNEPH  analyses.  Unfortunately,  these  data  arc  only  available  for  a  few  locations  in  the 
United  Slates,  where  the  resolution  of  the  RTNEPH  cloud-cover  analyses  are  poor  due  to  the  use  of  the  airways 
reporting  code.  It  might  be  advantageous  for  the  RTNEPH  model  to  record  cloud  cover  over  the  U.S.  using  only 
satellite  data  if  an  airways  report  contains  scattered  or  brcAen  conditions.  In  addition,  we  believe  testing  of 
observer-based  TCC  is  inadequate.  Perhaps  AWS  might  work  with  NWS  to  acquire  synoptic  reports  from  or  near 
WSl  sites.  This  would  enable  significantly  enhanced  studies  of  observer  precision. 

The  objective  of  this  study  was  to  quantitatively  evaluate  the  total  sky-cover  analysis  and  determine  its  precision.  It 
suggests  that  while  the  overall  quality  of  the  RTNEPH  total  sky-cover  analysis  is  good,  that  conclusion  is  highly 
dependent  on  the  operational  needs  of  the  data  user.  Figures  6-9  show  the  frequency  distribution  of  differences  (or 
errors)  between  RTNEPH  and  surface  reports.  Confidence  limits  of  the  data  can  also  be  calculated  using  these 
graphs.  Using  Figure  6,  we  can  state  that  the  RTNEPH  analysis  is  accurate  to  within  *1-  2  octas  69%  of  the  time 
averaged  over  (he  globe  in  winter.  However,  we  can  also  stale  that  it  is  totally  inaccurate  (error  of  7  or  8  octas) 
I2.S%  of  the  time.  The  acceptability  of  these  accuracies,  as  well  as  the  errors,  is  user-dependent.  In  general, 
however,  there  was  a  close  agreement  between  RTNEPH  data  and  surface  observations  except  in  the  polar  regions. 
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ACRINABS 


ACRINABS 

Acronyms,  initialisms,  and  abbreviations 

AFGWC 

Air  Force  Global  Weather  Central 

AWS 

Air  Weather  Service 

Neph 

Nephanalysis 

NH 

Northern  hemisphere 

NM 

Nautical  Mile 

RTNEPH 

Real-Time  Nephanalysis 

SH 

Southern  hemisphere 

SSM/! 

Special  Sensor  Microwave  Imager 

TCC 

Total  cloud  cover 

3DNEPH 

3-Dimensional  Nephanalysis 

WMO 

World  Meteorological  Organization 

WSI 

Whole  Sky  Imagery 
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